Assessing parental comprehension of online resources on childhood pain

We aimed to examine the patient education materials (PEMs) on the internet about “Child Pain” in terms of readability, reliability, quality and content. For our observational study, a search was made on February 28, 2024, using the keywords “Child Pain,” “Pediatric Pain,” and “Children Pain” in the Google search engine. The readability of PEMs was assessed using computer-based readability formulas (Flesch Reading Ease Score [FRES], Flesch-Kincaid Grade Level [FKGL], Automated readability index (ARI), Gunning Fog [GFOG], Coleman-Liau score [CL], Linsear Write [LW], Simple Measure of Gobbledygook [SMOG]). The reliability and quality of websites were determined using the Journal of American Medical Association (JAMA) score, Global Quality Score (GQS), and DISCERN score. 96 PEM websites included in our study. We determined that the FRES was 64 (32–84), the FKGL was 8.24 (4.01–15.19), ARI was 8.95 (4.67–17.38), GFOG was 11 (7.1–19.2), CL was 10.1 (6.95–15.64), LW was 8.08 (3.94–19.0) and SMOG was 8.1 (4.98–13.93). The scores of readability formulas showed that, the readability level of PEMs was statistically higher than sixth-grade level with all formulas (P = .011 for FRES, P < .001 for GFOG, P < .001 for ARI, P < .001 for FKGL, P < .001 for CL and P < .001 for SMOG), except LW formula (P = .112). The websites had moderate-to-low reliability and quality. Health-related websites had the highest quality with JAMA score. We found a weak negative correlation between Blexb score and JAMA score (P = .013). Compared to the sixth-grade level recommended by the American Medical Association and the National Institutes of Health, the readability grade level of child pain-related internet-based PEMs is quite high. On the other hand, the reliability and quality of PEMs were determined as moderate-to-low. The low readability and quality of PEMs could cause an anxious parent and unnecessary hospital admissions. PEMs on issues threatening public health should be prepared with attention to the recommendations on readability.


Introduction
Pain is a disturbing problem for both children and their parents.Pain can impair quality of life of both the individual and family.Untreated pain can result with chronic pain and chronic pain prevalence has been reported between 11% and 53% in children. [1]Pediatric pain can result with anxiety, depression, low self-esteem, the child can miss school and social or sportive activities.It can also create great concern and desperation among families because they may not know the cause of the pain or how to cure it. [1]lmost every individual has a smartphone or computer so internet is generally the first option for people to take health information. [2]This fact increases the number of healthrelated websites.It has been reported that there were more than 70000 health-related websites. [3,4]Internet is an important and valuable source for parents for searching health information.The 2018 Health Information National Trends Survey reported that internet is the first choice of health information resource for 74.5% of their respondents. [5,6]But the quality and readability of internet health information resources varies.Patient education materials (PEMs) should be readable and understandable for public.National Institutes of Health, American Medical Association and The US Department of Health and Human Services recommend that internet-based PEMs should be below sixth-grade level. [7]But unfortunately, several studies that investigate the readability of health information resources showed that many of them are higher than recommended level. [8]arents seek for online information which is easy to access, up to date and generally from a hospital website.
It has been shown that 1 of 8 parents first search internet before going to a hospital emergency room. [8,9]This finding shows that, it is important to conduct readable and understandable online health information.Especially for parents that are concerned with their child health, it is more important to understand the PEMs to lower the anxiety, unnecessary hospital admissions or the risk of underestimating serious conditions.However, we did not find any study assessing the internet-based PEMs on Child Pain, which is a very important issue for public health.We hypothesized that the readability, quality and reliability of internet-based PEMs related to pain in children were higher than the recommended levels.In order to test this hypothesis, we evaluated the levels of these parameters of internet-based PEMs related to Child Pain.

Methods
We designed this cross-sectional, observational study to evaluate the readability and quality of internet-based PEMs about child pain.After approval of Dokuz Eylul University Hospital noninvasive Studies Ethics Committee (2024/08-10), on February 28, 2024, the terms "Child Pain" "Pediatric Pain" and "Pain in Children" were searched on Google (https:// www.google.com)search engine.Google search engine was preferred and used because it is the most preferred search engine. [10]n order to avoid search results being affected, before our search in Google search machine, web browsing history was erased and cookies were deleted.At the same time, we signed out from all Google accounts.Google Incognito mode was used.13][14] The websites that are not related to pain in children, has advertisement marking, need subscription or registration, are not in English, the scientific papers which do not include patient information, the sale sites that do not have a patient information section, the ones include video or audio content without a text and duplicate websites were not included in our study.Video contents, figures and tables, pictures, punctuation marks, website URLs, references, telephone, address and author information in the text were also excluded while evaluating the texts in order to prevent false results. [11,12,15]f no evaluation criteria were on home page of the website, the 3-click rule was used.The 3-click rule is; the user should reach information with up to 3 mouse clicks, it is thought that unless they reach information in 3 clicks the users leave that website. [11,12,15]he readability and quality were evaluated by 2 independent researchers (V.H., I.E.) with proper measurement tools.The data was evaluated by 2 different researchers independently, after training how to use the measurement tools, to avoid bias.

Reliability and quality of websites
]19] It assesses 4 items; authorship, attribution, disclosure and currency.The presence of each item adds 1 point to the score.JAMA Quality test criteria, Modified DISCERN and Global Quality Score (GQS) were used to assess the quality of the websites as previous studies. [11,12,20]Modified DISCERN includes 5 questions addressing the quality of the website and each "yes" answer is 1 point where higher scores indicate higher reliability and quality.GQS is a 5-point scale where 1 point is poor quality and 5 points is excellent quality.

Readability
The texts of the websites were transferred to Microsoft Office Word 2007 (Microsoft Corporation, Redmond, WA).In order to analyze the readability, as in previous studies about readability, the punctuation marks, excluding periods, were erased. [11,12,17]The texts were evaluated by using "www.readability-score.com"and the formulas; The Flesch Reading Ease Score (FRES), Flesch-Kincaid grade level (FKGL), Simple Measure of Gobbledygook (SMOG), Gunning Fog (GFOG), Coleman-Liau score (CL), automated readability index (ARI) and Linsear Write (LW) were used to assess readability. [11,12,17]All formulas were used for each PEM in our study.These are the most popular formulas used while assessing the readability of texts and the ones commonly used in similar articles. [11,12,17]FRES formula uses the ratio of total number of words to total number of sentences and the ratio of total number of syllables to total number of words.FKGL analyzes the sentence length and word complexity.Longer sentences and words with more syllables indicate a higher difficulty level.The SMOG Index focuses on polysyllabic words, which often indicate complex vocabulary and sentence structures.The GFOG formula calculates a grade level based on the average sentence length and the percentage of complex words.CL focuses on the average number of letters and sentences, assuming that longer words and longer sentences are more difficult to read.The ARI formula assesses the average number of characters per word and the average number of words per sentence.LW formula was created by The US Air Force to evaluate the readability of their technical manuals.It calculates the grade level of the text by evaluating the sentence length and the number of words with 3 or more syllables.The readability scores were compared to sixth-grade readability level. [11,12]This readability level is ≥ 60.0 for FRES formula and < 7 for other used formulas. [11,12]

Content analysis
Contents of PEMs about child pain were evaluated.In the contents; definition, pathophysiology, incidence, epidemiology, risk factors, diagnosis, treatment, pain scores, morbidity, mortality, complications and prevention were evaluated. [11,12]

What known
Internet is widely used for health information by individuals.However, internet-based patient education materials are usually difficult to understand by general population.

What new
We evaluated the readability and reliability of websites related to child pain.We found that the readability of websites is below recommended level of sixth-grade level.Most websites had a reliability and quality level of moderate-to-low.www.md-journal.com

Table 1
Comparison of Blexb, JAMA, DISCERN, GQS scores, presences and reading levels according to the typologies of the websites.<.001

Society/Professional
JAMA Median (min-max) .001 .021 Insufficient data (0/1 point) Correlation analysis was made with Spearman correlation coefficient.P value of <.05 was determined as significant."Bonferroni adjustment" was used in multi-group analyses, and the P value was determined by the number of groups.

Results
The search using Google showed 1.5 million results.The records of the first 167 websites were included in our study.After the first 167 websites, the duplications started.20 websites (12%) including scientific materials, 6 (3.6%) unrelated websites, 4 (2.4%) commercial websites, 4 (2.4%) websites without texts and 36 (21.6%) websites that could not be reached were excluded from the study.96 websites that met the inclusion criteria were analyzed.

Website typologies
In 96 websites included in our study, hospital websites (n = 44, 45.8%) have the highest percentage, the second most common typology was health-related websites (n = 18, 18.8%) (Table 1).

Reliability and quality evaluation
The JAMA score for 96 websites was 1 (0-4), GQS score was 3 (1-5) and DISCERN score was 2 (0-5).75 (78.2%) of 96 websites' GQS score was ≤ 3, JAMA score of 86 (89.5%) websites was ≤ 2 and DISCERN score of 64 (66.7%) websites was ≤ 2 indicating that they have a moderate-to-low reliability and quality.We could not find any difference between typology of the websites with GQS and DISCERN scores.But there was a statistically significant difference with JAMA score (P = .009)(Table 2) (Fig. 2).The quality of websites with JAMA score can be aligned as; health-related > society/ professional organization > hospital > news > other > government websites.

Correlation analysis
We found a weak negative correlation between JAMA score and Blexb score (P = .013)but we could not find any correlation with GQS and DISCERN scores (P > .05).A weak correlation was found between LW score and Blexb score (P = .043).No correlation was observed between Blexb and other readability scores (P > .05)(Table 3).

Content analysis
Each topic related to child pain was assessed in content analysis and their frequencies were; definition 38.5%, pathophysiology 10.4%, incidence 11.5%, epidemiology 22.9%, risk factors 26.0%, diagnosis 68.8%, treatment 72.9%, pain scores 22.9%, complications 24.0%, morbidity 9.4%, prevention 19.8%.Mortality was not mentioned in any of the websites.We did not observe a significant relation between website typologies and contents (P > .05)Table 4).

Comparison of the first 10 and other websites
The first 10 websites' Blexb rank values were not different from the rank values of other 86 websites (P = .891).Also, we could not find any difference between the typologies of the first 10 and the other 86 websites (Fig. 3).
When the quality of first 10 websites was compared with other websites we found a statistically significant difference in GQS score (P = .02).Similarly, there was a significant difference in GQS quality score distribution between the first 10 websites and others (P = .02).The JAMA and DISCERN scores of first 10 websites were similar to other 86 websites (P = .435and P = .282,respectively).The distribution percentages of JAMA and DISCERN scores of first 10 websites and others were not different (P = .761and P = .397,respectively).
Treatment (n = 8), diagnosis (n = 7), and complications (n = 5) were most common contents in the first 10 websites.We could not find any difference between the contents of first 10 websites and the others (P > .05)(Table 5).

Discussion
We found that the readability grade level of internetbased PEMs about "Child Pain" was significantly higher than the recommended sixth-grade level.We also showed that most of the PEMs' reliability and quality levels were low to moderate.Hospitals' websites were the most frequent websites and there was not a difference between the first 10 and other websites typologies and readability.
Pain is an unpleasant feeling that can be associated with tissue damage. [21]It can be in an acute or chronic nature.In Germany, the prevalence of pain has been reported as 64.5% for children between 3 and 10 years and 77.6% for 11 to 17 years. [22]The incidence of chronic or recurrent pain has been reported between 11% and 53%. [1]These results show us that more than half of the children suffer from acute or chronic pain.In a recent review, the authors summarized the epidemiologic studies and found that one-third of the children had stomach aches and 20% of the children had inadequate pain relief therapy after surgery. [21]It has been shown that pain in pediatric age can result with impairment of cognitive and social development. [23,24]Pain experience can be stressful and even traumatizing for the child and parents.Anxiety, depression and low self-esteem can arise as a result of pediatric pain.Pain is also the reason of concern and desperation of the child and parents. [1]owadays internet is the primary source of health information for many individuals. [25]It was shown that 90% of the people who live in America use internet and health-related topics constitute 72% of internet searches. [26]Nearly, 6.5 million Google searches are for accessing health information per day. [27]or anxious parents searching internet for information about their child pain, the internet-based PEMs should be easy to read and understand.Otherwise, despite easy access to health information on internet, PEMs can cause confusion and anxiety. [28]specially for children suffering chronic pain, improving parent related factors is one of the key factors to improve child functioning. [29,30][33][34] Parent behaviors are usually related to their experiences and the information they have.In a recent meta-analysis, correlation between parent anxiety and child pain and disability has been shown. [34]t is understandable that a misinformed parent would be more anxious and protective.This study showed that the PEMs about child pain are difficult to read and have moderate quality which could easily result with misinformed parents.Therefore, the easiest way for parents to access information, the internet-based PEMs, should be easy to read and understand.
Although the number of internet-based PEMs is increasing due to high demand, their readability and quality varies. [4]oreover, the understandability of PEMs depends on the person health literacy which is defined as to read and understand health information and use it while making health-related decisions.The adequate health literacy has been reported as 12% in American population. [35]As long as the readers do not understand, the information and accuracy of the website are not important.When the patient in question is a child, the parents have to make a health-related decision on behalf of the child.This would increase the anxiety of the parents.As we determined in our study the PEMs about child pain are difficult to read so it is questionable that the PEMs could help an anxious parent to make health-related decisions.This is an important issue for children health and also for healthcare sector because a misinformed parent can decide to go to emergency for unnecessary pain or decide to stay home while urgent care is needed.
Readability is an objective measurement of written material understandability.There are several formulas to evaluate readability but the best one/ones to evaluate health-related materials are unknown. [36]Therefore, in our study, we used 7 different formulas to evaluate the internet-based PEMs about "Child Pain" and also the estimated grade level.The recommended grade for readability is sixth-grade level.The United States Department of Health and Human Services considers below sixth-grade level as 'easy to read', 7 to 9 th grade level as "average difficulty" and above 9th grade level as "difficult." [37]This study showed that most of the internet-based PEMs about "Child Pain" are above sixth-grade level with all readability formulas except LW.Similar to our findings Arslan et al [2] investigated the readability of PEMs about chest pain in children and found that their between first 10 and websites similar to our findings.Contrary to our findings, Basavakumar et al [13] reported in their study that the first 10 websites were more readable than other websites.The GQS score showed that the quality of first 10 websites were higher than other websites.An interesting finding of this study was we could not find a difference at rank scores of first 10 and other websites.This finding may be result of low readability and quality of first 10 websites.
The most common contents of the PEMs about "Child Pain" were determined as "diagnosis" and "treatment."This is an understandable finding since the most curious topics about pain are; the reason of the pain and how to treat it.
There are several limitations of this study.The first limitation is an unavoidable one and is related to internet nature.Internet is a dynamic information source so the ranks and search results may change.Second limitation is, we only search the websites in English.Including all languages would result in different results.Third and also an unavoidable limitation of our study is even though we cleaned the history and cookies of the search engine, the search results could still be related to our geographic location.In different regions of the world, the same keywords could result with different websites.Another limitation is evaluating readability with computer-based formulas.These formulas may overestimate the readability of the evaluated material.Although many studies assess the readability of PEMs with computer-based formulas, the ideal formula to evaluate readability of internet-based PEMs is unknown. [2,4,5,7,8]We used 7 computer-based formulas to reduce the misleading results.The last limitation of our study is, we used only general search terms about child pain like "Child Pain," "Pediatric Pain" and "Pain in Children."We did not use the keywords for specific pain types like headache or postoperative pain because we wanted to include all kinds of pain that children may suffer and evaluate the PEMs about them.Our aim was to assess the readability and quality of PEMs about pain in childhood.Further studies could be planned to evaluate the PEMs about specific pain types in childhood.
In conclusion, we evaluated the readability and quality of internet-based PEMs and showed that the reliability and quality of websites are moderate-to-low and readability grade level is higher than the suggested sixth-grade level.We also found that the rank of first 10 websites did not differ from other websites and we believe that this could be as a result of low readability and quality of the websites.Our findings show that the PEMs that parents search for information about their child health are difficult to read and understand.We believe that while preparing PEMs, the readability and health literacy of general public should be considered by the editors of the websites.This would help parents to find satisfying information about their child health and decision-making.Further studies may be needed to explore the readability of PEMs about specific pain types that may affect a child well-being.

Figure 2 .
Figure 2. GQS, JAMA and DISCERN scores according to website typologies.GQS: Global Quality Score, JAMA: Journal of American Medical Association.

Figure 3 .
Figure 3. Website typologies distribution of top 10 and other websites.
Liau score, FKGL = Flesch-Kincaid grade level, FRES = Flesch reading ease score, GFOG = Gunning FOG, GQS = Global Quality Score, JAMA = Journal of American Medical Association Benchmark Criteria, LW = Linsear Write, SMOG = Simple Measure of Gobbledygook.SPSS Windows 25.0 (SPSS Inc., Chicago, IL) was used for statistical analysis.Frequency data were given as numbers (n) and percentages (%).Data taking continuous values were represented as median (minimum-maximum).Frequency variables were compared with Chi-square or Fisher exact tests.The data taking continuous values were assessed with Kruskal-Wallis or Mann-Whitney U tests.

Table 2
Top 10 and other than top 10 websites' mean reading indices and statistical comparison of text content to 6th-grade reading level, Blexb rank, JAMA, DISCERN, GQS, and typology results.
Bold character; statistically different (P < .05).ARI = automated readability index, CL = Coleman-Liau score, FKGL = Flesch-Kincaid grade level, FRES = Flesch reading ease score, GFOG = Gunning FOG, GQS = Global Quality Score, JAMA = Journal of American Medical Association Benchmark Criteria, LW = Linsear Write, SMOG = Simple Measure of Gobbledygook *Comparison of the first 10 websites and remaining 86 websites according to parameters.**Comparison of the 96 websites' according to 6th grade reading level.

Table 5
Content analysis of websites about child pain.